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INTRODUCTION 


Increasingly, attainment of a postsecondary credential is seen as a prerequisite to holding a 
quality job in the rapidly changing economy. Jobs typically requiring a postsecondary cre- 
dential are projected to account for 44 percent of new jobs in the 2008-2018 time period.” In 
addition, the Obama administration seeks to “[lead] the world in the percentage of Americans 
with postsecondary degrees and/or industry-recognized certificates ... by 2020.”° The recent- 
ly passed Workforce Innovation and Opportunity Act (WIOA) bipartisan, bicameral legisla- 
tion reauthorizes the Workforce Investment Act (WIA) for six years, from 2015 through 2020. 
WIOA establishes as some of the core indicators “the percentage of participants who obtain 
recognized postsecondary credential, a secondary diploma or equivalent as well as measur- 
able skills gains toward such a credential to be used to gauge the success of the wide range 
of publicly funded activities that focus on workforce development. 


But even as policy makers have placed an increasing focus on training and preparing work- 
ers to obtain postsecondary credentials, questions remain about the value to various actors, 
especially for job seekers and workers who often invest time and money to obtain a creden- 
tial with the hopes of improving their employment prospects and raising their earnings. This 
report begins to address this issue. Drawing on data from the randomized control trial of 
the Sectoral Employment Impact Study (SEIS)*, which included three sector-focused training 
programs, we explore whether individuals who obtain credentials work more and earn more 
over time than they would without such credentials. In Section 1, we describe sector-focused 
training programs, the individuals in the study. In Section 2, we describe the specific creden- 
tials provided to the program participants and explore the value of credentials using a statisti- 
cal technique called instrumental variable (IV) and highlight three findings. Section 3 outlines 
conclusions and some essential caveats. All values stated in this report are in 2014 dollars, 
unless explicitly noted. 


'We thank Chris King, senior research scientist and director, Ray Marshall Center for the Study of Human Resourc- 
es at the University of Texas at Austin, and Burt Barrow, Amsterdam Professor of Public Service and Economics at 
The Trachtenberg School of Public Policy and Public Administration at George Washington University, for providing 
useful comments. 


? Bureau of Labor Statistics, Employment Projections 2012-22 , December 19, 2013 http://www.bls.gov/news. 
release/pdf/ecopro.pdf (accessed 12.14.2014) 


3 Employment and Training Administration. 2010, December 15. Training and Employment Guidance Letter No. 
15-10: 1. Employment and Training Administration. 2012, January 13. Upcoming ETA Funding Opportunities. 1-2. 
http://gov.georgia.gov/00/press/detail/0,2668,165937316_165937374_181836313,00.html http://cre.virginia.gov/ 


4 Sheila Maguire, Joshua Freely, Carol Clymer, Maureen Conway and Deena Schwartz. (2010). Tuning In to Local 
Labor Markets: Findings from the Sectoral Employment Impact Study. Philadelphia, PA: Public/Private Ventures. 
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SECTION 1 


A REVIEW OF THE SECTORAL EMPLOYMENT STUDY (SEIS): 
ITS PROGRAMS, PARTICIPANTS AND IMPACTS 


The Sectoral Employment Initiative Study (SEIS) is the first rigorous random assignment eval- 
uation of sector-focused training programs that prepare low-income, unemployed, under- 
employed and underskilled workers for skilled entry-level positions, and connect them with 
employers seeking to fill such vacancies.° 


The three sector-specific trainings programs selected for SEIS were: 


e¢ Wisconsin Regional Training Partnership (WRTP), serving the Milwaukee area, which offered 
short-term training programs in the construction, manufacturing and health care sectors; 


e Per Scholas (PS), serving New York City, which offered computer technician training for 
entry-level jobs in repairing and maintaining personal computers, printers and copiers, as 
well as installing and troubleshooting computer networks; 


e Jewish Vocational Services (JVS), serving the Boston area, which offered medical billing 
and accounting training. 


To be included in the study, programs must have been operating for at least three years, 
tracked outcomes for several years and showed wages at or above $8 an hour (equivalent 

to $10 in 2014 dollars). SEIS sought to determine whether sector-focused training programs 
could increase the earnings and employment of low-income workers and job seekers, includ- 
ing those with potential barriers to employment, such as irregular work histories, homeless- 
ness, criminal records, welfare receipt or education levels. 


Programs included in the study served a range of disadvantaged job seekers. Table 1 shows 
the individuals who applied to the sector-focused programs during the study’s enrollment pe- 
riod in 2003 and 2004. Half the applicants were women. Most applicants were non-white and 
relatively young; the average age was 32, the median was 30, and a little more than a quarter 
were under 25. While 34 percent were employed either full time or part time at the beginning 
of the study, only 10 percent had been employed full time during the entire 12 months prior 
to the baseline survey. On average, applicants had earned only $9,872 in 2004 dollars (or 
$12,341 in 2014 dollars) in the prior year. Nearly 40 percent had received public assistance 
(ever), including the 23 percent of participants who were on welfare at the time of enrollment. 
About one in five applicants had been convicted of a crime. Seven percent had been home- 
less in the year before applying to the program. In terms of education, 53 percent had a high 
school diploma only and another 22 percent had obtained a GED; 8 percent had an associ- 
ate’s degree and 9 percent had a bachelor’s or master’s degree. 


° Sheila Maguire, Joshua Freely, Carol Clymer, Maureen Conway and Deena Schwartz. (2010). Tuning In to Local 
Labor Markets: Findings from the Sectoral Employment Impact Study. Philadelphia, PA: Public/Private Ventures. 
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A lottery was used to select which eligible program applicants would be included in the pro- 
gram group and which became part of the control group.® Two years later, applicants in both 
groups were re-contacted and surveyed. Table 1 shows that the two follow-up groups were 
comparable on all characteristics except three: Program group members were slightly more 
likely to be married and to be foreign-born and slightly less likely to have ever been incarcerat- 
ed. These three variables are included in all analyses to control for these differences between 
the program and control groups.’ 


The study found that over this two-year period, the sectoral programs had had substantial 
positive impact on earnings across all types of participants. Table 2 presents the experimental 
impact estimates in 2014 dollars (Maguire et al. 2010, p. 11-13). The experiment found that 
program group individuals earned more over the 24-month period, were more likely to obtain 
and maintain employment, and were more likely to be well paid — i.e., make almost $13 an 
hour (in 2014 dollars) or more — than controls, particularly in the second year. Key outcomes 
include the following: 


e The program group members earned $5,295 or 18 percent more than controls, over the 
24-month study period. Most of the increase occurred in the second year ($396 per month, 
or 29 percent more) given that the program group was in training during the first year. 


e Program group members were more likely to work than controls, working on average with 
1.3 months more on average over the 24-month study period. During the second year, 
52 percent of the program group was working all 12 months in the second year versus 41 
percent of the controls. 


e Program group members were more likely to work in jobs with higher wages and in jobs 
with benefits (such as health insurance and paid sick leave). For instance, program group 
members worked two more months than controls in jobs that paid at least $13 an hour 
during the two-year study period. 


6 Throughout this paper, we use the term “program group” to mean the “treatment group.” It is the group of individ- 
uals who were offered the chance to enroll in the program. 


’ The follow-up survey sample included 1,014 respondents, 75 percent for the control group and 82 percent for 
the program group. To determine if the program and control groups were still comparable, we regressed being 

in the program group as a function of all the baseline characteristics in Table 1. None of the individual covariates 
were significant and the joint test that they were all zero could strongly be accepted, indicating that the two groups 
still were comparable. We also test the comparability of each characteristic individually using either a t-test or a 
chi-squared test. These test showed that with the exception of three characteristics — being married, being for- 
eign-born and having been incarcerated — the groups were the same. To be cautious, these and the other baseline 
variables were controlled for in all subsequent analysis. 


8 The findings in the earlier report are based on an “intent to treat” analysis, i.e., all participants assigned to the 
program group were included, even if they did not attend or complete the program. In this study, 7 percent of those 
offered a position in a sector-focused training program never attended, and 41 percent of those assigned to the 
control group received other job training services, including job-specific skills training, job search assistance, train- 
ing in basic reading and math skills, internships, on-the job training and GED classes. 


BASELINE CHARACTERISTICS OF SEIS FOLLOW-UP SAMPLE 


TABLE 1 


Total Control Program 
Group Group 
Sample Size 1014 485 529 
Gender 
Male 47% 49% 46% 
Race/Ethnicity and Foreign-Born Status 
African-American 60% 61% 59% 
Latino 21% 23% 20% 
White 12% 11% 13% 
Other 6% 5% 7% 
Foreign-born 23% 21% 26%*™* 
Age (average) 32.2 32.0 a2 
18 to 24 28% 29% 27% 
25 to 54 70% 68% 71% 
55 and Older 2% 3% 1% 
Education 
More than High School Diploma 18% 17% 19% 
High School Diploma 53% 54% 53% 
GED 22% 21% 22% 
Less than High School Diploma 7% 7% 6% 
Other Characteristics 
Married 18% 15% 20% ** 
Ever on Welfare 37% 36% 38% 
On Welfare at Baseline 23% 23% 22% 
Has Access to a Vehicle 45% 44% 47% 
Average Number of Children in Household 1.2 1.2 1.3 
Move in Last Two Years 43% 41% 44% 
Completed Other Training Before Baseline 25% 27% 23% 
Homeless in Year Prior to Baseline 7% 7% 7% 
Ever Convicted of a Crime 22% 24% 20% 
Formerly Incarcerated 17% 20% 15%* 
Employment History at Baseline 
Average Months Employed Year Prior 6.8 6.7 6.9 
Employed (Part-time or Full-Time at Baseline) 34% 33% 34% 
Worked Full-Time all 12 Months Prior 10% 10% 11% 
Avg. Months Working Full-Time Year Prior 3:5 3.4 a 
Total Earnings Year Prior to Baseline (2004 $) $9,872 $10,171 $9,599 


Note: Stars indicate that the program group was not statistically the same as the control group on this characteristic. 


* indicates that the level of significance is 0.10 or less, while ** indicates a 0.05 level of significance. 
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e Positive impacts were seen for all subgroups examined, namely subgroups defined by 
gender, race/ethnicity, criminal records, welfare receipt, youth/young adults and immi- 
grants. The impacts were the largest for Latinos, foreign-born individuals, ex-offenders 
and women. 


While each program included in the study employed a unique strategy, focused on differ- 

ent industries and occupations, and crafted its program to respond to local circumstances, 
through site visits, focus groups and interviews, researchers identified common elements 
shared by the three programs: a strong link to local employers that results in an understanding 
of the target occupation and connections to jobs; recruitment, screening and intake processes 
that result in a good match between the applicant, the program and the target occupation; 
individualized services to support training completion and success on the job; job readiness, 
basic skills and hands-on technical skills training offered through the lens of a specific occupa- 
tion or sector; and strong organizational capacity with the ability to adapt. 


TABLE 2 


EXPERIMENTAL IMPACTS ON EMPLOYMENT AND EARNINGS 
TOTAL SEIS SAMPLE, ALL SITES 


Months Employed 


24 Months Laer 
Hours Worked 

24 Months 245*** 
Ever Worked a Job Paying $11 an Hour or More 

24 Months 14%*** 
Total Earnings 

24 Months 65,295°"" 


Asterisks (*) indicate statistically significant differences between program and control groups. ***p < 0.01 
y sig prog group p 


These are impressive findings. They suggest that when well-designed sector-focused training 
programs work closely with an industry or sector to determine and teach the skills workers 
need to know to be employed in that sector, low-income workers can significantly benefit. 


SECTION 2 


THE VALUE OF CREDENTIALS: 
ANALYSIS OF THE SEIS DATA 


The original study of SEIS indicated that these three sector-focus programs had substantial 
positive impacts on participants’ earnings. This earnings impact undoubtedly is due, at least 
to some degree, to the technical or specific skills participants learned in the program. How- 
ever, as we will see below, many of the participants also earned credentials that are valued 
by employers within their local labor markets. Perhaps the participants needed to earn these 
credentials to prove to or “signal” to the local employers that they indeed possessed the 
needed skills at the appropriate level of competence. If this were the case, the earning im- 
pacts would be mediated through the credential. In other words, for participants to benefit, 
earning a credential may be a necessary, indeed a critical signaling step. 


ANALYSIS APPROACH 


Because many individuals (in the program group and controls) in the SEIS data set earned 
credentials during the study period, some might think that a simple way to investigate the 
value of a credential would be to compare the earnings of credential holders with other indi- 
viduals who are like the credential holders demographically (similar age, education, race, etc.) 
who did not earn a credential. Unfortunately, this simple regression approach does not work 
because it is quite likely that credential holders are the most motivated individuals and they 
may well have earned more even without a credential. In other words, part of the earnings 
differences we observe between credential holders and non-holders is likely due to motiva- 
tion or persistence, not just the credential. 


To deal with the problem, we examine the role credentials have on earnings by using a statis- 
tical method designed to “cleanse” estimates of these motivational biases called Instrumen- 
tal Variables (IV) (Angrist, Imbens, & Rubin, 1996; Heckman, 1996, 1997; Gennetian, Morris, 
Bos & Bloom, 2005)? Angrist et al. (1996) shows that experiments in which individuals are 
randomly assigned to a program or control group provide an ideal situation to use this tech- 
nique if treatment status is correlated with the variable of interest, in this case a credential. 
Indeed, we will see below that treatment status is strongly correlated with credential holding. 
Intuitively, this technique compares the earnings of controls who did not earn a credential 
with the earnings of program individuals who would not have “naturally” earned a certificate. 
The difference in earnings between these two subgroups is then attributed to the difference 
in the rate of credentials. 


* In particular, we will use the treatment status dummy as the instrument for having a credential per Heckman, James 
J."Randomization as an Instrumental Variable,” Review of Economics and Statistics, 1996, v78 (2,May), 336-341. 
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Below, we first describe how we collected data about credentials in the SEIS. Then we show 
that being in the program group is strongly correlated with having a credential. Next, we ex- 
amine evidence that indicates that the impact of the sector training programs worked through 
helping participants earn a certificate when they otherwise would not have. Then we examine 
the impact of obtaining a credential on earnings, first overall and then by the type of creden- 
tial. In each section, we lead with the main finding, and then provide our evidence. 


t-Tar-tol Co) f-S-ol aa) oLUic-Tmc-rotalalrerr-lamac-lialiarem olcevele-laam oleenilel--melk-t-lenZ-laie-lel-tom et-Yole)(-MyUiaa 
italoWd.4] Mm al-X-Ko(-Yo Mi co MciU(xet-\-Yo MN] [0) o)-MNC-[-1c-Yo McoM alc) of-1] ae-late Mi aat-lialc-lar-laleMolmm el-1 acre) are] 
computers, printers and copiers, as well as the installation and troubleshooting of com- 
relULC-Y amar -1aUVold <omm | Meo) ic-1¢-Yo mm of-lad Col] of-lal cco al-We) o) efeladelaliavan com (1. <M - WAG am -> <li lam alte 
farcleKear-] Malco) aaat-licelame-tovalare Coxe hmaN MD me-ladiiler-\atela dal anY(-lani( Com colUlarel-\a(o)ara(-\V-1ML <M cole 
entry-level work in PC installation, preventative maintenance, networking, security and 
troubleshooting, and IT customer service skills. The A+ is sponsored by CompTIA, a non- 
redeeyiimag-Le(-Maal-Van] el-Vecval] oM-lsscefeit-| (ola mant-le (ov) oMolm UMN oll al-t-x--S-¥r- Alem eco) (Sle) al-] Halts 
was established in 1993 as a “vendor neutral” certificate to standardize the skills of IT's 


iF-Yoi me] coy Ulate Mie) ace) (-Wam Kolel-WA im Meteo [Ullc-Yom (olar-We-lale[-Me)meteeit] of-1d(e)aice-lale MM UIt-To Ml oN ar-1A) 
Takerx-¥-s1l ave ualelenl ox-)ameymm-v00] ©) (e)Z-16-Me lam all alare mel -Yetkst(o)almr-Lol ceyss-ar- IVA [e(-Mae-lale [-Me) m-Ta0) ©) oN 2165 
including Dell, Intel, and Lenovo service techs and CompuCom and Ricoh. 


The Wisconsin Regional Training Partnership (WRTP) is an association of employers 
Taro Mel al Colat-vanW Zel(olaMect-X-1, <M CoM -ie-V[aMe-1aleme-laac-Leim elle] ann\z-Ye[-M (o]os-MlaMm\VIIM UclUL <-Y-Yar-Vale Meta t-l Cc) 
for-] g-1-J ame) 0) oe) quel alin(-SMm re) am (earl areceant-Mr-lalemmUlat-Janl ellen i-Yo mecolaalanlelelinvanc-ss(el-Yal cm isl eel -te 
velops training programs (generally lasting between two and eight weeks) in response 
i CoM) oX-Yoih i (om Van] ol(oy\{-Vecwm c-Yo LU[-S<1 emo) am coMei(-t-Va Vane (-aidii(-Yom[-] oXe) mant-1a.<-1 Mm al-\-Yo em Lcmyale) gcc c-1aan) 
redc-ex-danl odCoynal-laiamec-lialiare im olcete] clas maim aar-meo)aliagu(ei Cola manr-laleiicvecelalare Mr-laveMat-t-1 ida Mmer-1¢— 
sectors were included in the study. WRTP also provides training to incumbent workers. 


WRTP offered participants the opportunity to prepare and sit for the state administered 
exam for Certified Nursing Assistant (CNA) and Certified Medical Assistant (CMA). CNA li- 
ox-VariUl¢-MiMa Yo [0]1c-Yom cola (ol osmlale-Me-lalel-Mevmlasiuiaeieeyar-]mx-inilave [ml aver U(ollaremuat-Me-]e)(e|\Va-).4ex-1alens 
Take Me-VareMevat-lave | lave fm atedaat-taai-t-1(aMer-1x-M--Yoico) em Mat -W(-Yo(-1¢-]M-]aloMcit-1\-Meloh-laaleat-lalMaar-larer-li-) 
training requirements for service providers under Medicare and Medicaid with the federal 
reo NY-Tanlaat-vaiacc-ltdlare Maalialioalelsacmcolmiclialiarem-laremaat-Mcit-1c-Meym UU txecolasiame-telllialave im PACH aTeLUlEs 
WRTP participants had an opportunity to earn several different certifications in the construc- 
tion field, including in asbestos removal, utilities construction and general construction. 


Jewish Vocational Service-Boston (JVS) is a community-based nonprofit that has pro- 
Nite [-Yo Nola are) cx- We (-0V/-1(o) o)aai-lal mcx-1 a (ex-1om (o) am gale) coM Nata AOMZ-y-1 e-em [aver Ulellave Me) el-Ye-lalale Me) al-) 
ro} dal ct @) al-tes) Ke) oN Or-1¢-1-1 mm OTe k-16-m ( @)al-tovice) os) MU lale(-Loll oN’ maal-MAVold are) ce-M aNZ-Siuaal-lal ar aCel 
AUTEN Tae dan Losi Cola ie-1c-t- Pan comar-llaliave Mm olcotele-laal-mlamant-rel fer] ollie M-lavem-Leexeleladlate MU-16—) 
included in the study. JVS provided participants with job-specific occupational skills 
idalcelete]aM-laMialc-larthU- Mom -eitarelaldamac-lialiarem olcere]c-laam-laremcom-ie] o) od(-lani-val a dal cm tc-lial lave my diane) 
alte lam (-M-)Rerm-10] ©) eXelamelUlalarem-larom-lic-1mcat-w olcote]e-laamam\y/(el-1m of-lad(ol] ol-1al <M)/-1e-Uvolaat-1aPm tale 1aNY 
fo) Mat -Tan Ma colelate MiVolant-laMmielaret-Xommtalcolele]aMmi-lir-1k-Ma-1celanamele| -1é-em @X-ladl ier] -\Mo) maxon] e) (- 
itfeyaMni-1e-m ol Ken Le(-Xomcolamilaliiallare fm tal-Sx-Mlalnclaar-l Nace) eL-1e-1c-Yemae-llallare Maatexel 0] (-cmN \eoM Vela <-to. 
foi foY-t-1 NARUYALaMm-]ae=Le NVA Xo) gYae [cole] oMey ama -x-11 (a lmer-1 cm -100] 0) (oN,-16-Madar-1 Mm OU aCe) Molla diil-Yo mm al-t-Te[-te 
competencies and reviewed curriculum. 


COLLECTING CREDENTIAL DATA IN THE SEIS 


Postsecondary credentials include an array of awards and degrees that require differing levels 
of formal academic and occupational training, lengths of study, and costs to learners.'? These 
credentials are often awarded by for-profit and nonprofit colleges and technical schools. 
Others, like those in the SEIS study, are earned to short-term training programs operated by 

a range of organizations including nonprofits, and labor-management partnerships. Some of 
these credentials may be classified as “industry-recognized,” are defined by the U.S. Depart- 
ment of Labor as “either developed and offered by, or endorsed by a nationally recognized 
industry association or organization representing a sizable portion of the industry sector, or a 
credential that is sought or accepted by companies within the industry sector for purposes of 
hiring or recruitment which may include credentials from vendors of certain products.” This 
definition covers a wide variety of credentials across a broad array of industries. 


Many of the individuals in the program group were offered the opportunity to prepare for 
and earn credentials. Among the three programs included in this study, two of them, WRTP 
and PS prepared participants for certifications and licenses that meet the USDOL definition 
of industry-recognized while the third, JVS, awarded a certificate for those participants who 
finished the program. Credentials offered through SEIS programs could be earned after 
relatively little training (5-15 weeks) and the training was delivered by the one of the three 
nonprofit SEIS organizations. 


In the SEIS follow-up survey, respondents were asked about their employment histories 
and whether they had received any vocational, technical or business credentials since they 
applied to the program. Those who had were asked to identify the type of credential from 
among the following choices: 


e A+/Networking/Computer Technician; 

e Clerical/secretarial; 

e¢ Medical office; 

e Accounting; 

e Certified Nursing Assistant (CNA); 

e Medical Assistant (CMA); 

e Construction - Utilities; Construction - Asbestos; Construction - Other; 
e¢ Commercial Drivers’ License (CDL); 

e Manufacturing; 


e Other. 


'0 The U.S. Census Bureau ‘s report , Measuring Alternative Educational Credentials: 2012, provides the first national 
estimates that show a sizable portion of the U.S. population holds alternative educational credentials independent of 
traditional college: 22 percent held a professional certification or license, and 9 percent held an educational certificate. 


The Census uses definitions developed by the Interagency Working Group on Expanded Measures of Enrollment and 
Attainment (GEMEnA), that is developing national measures for education and training credentials. These definitions 
provide an initial framework under the umbrella term of credentials. 
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Based on discussions with program staff in the three cities, the following subset of credentials 
were designated as “industry-recognized” since they are sponsored by an industry, trade or 
professional association, and/or require passing an examination sponsored by an association 
or leading firm in that industry, trade or profession: 


e A+/Networking/Computer Technician; 
e Certified Nursing Assistant (CNA); 

e Certified Medical Assistant (CMA); 

e Construction - Utilities; 

e Construction - Asbestos; 

¢ Commercial Drivers’ License (CDL). 


The other responses were designated as “program-generated/industry recognition uncon- 
firmed.” This latter group included all the program completion certificates provided by JVS, and 
other credentials that were not sponsored by an industry, trade or professional association. 


FINDINGS 


1. Attending the sector-focused training programs greatly increased the likelihood of gain- 
ing a credential 


Table 3 shows the distribution of credentials held by program and control group members. 
The data show that nearly half (46.5 percent) of all sample members obtained some type of 
credential. A quarter (109) of the credentials of the sample members were held by members 
of the control group, while three-quarters were held by program group members. Thus, 
attending the sector-focused training programs greatly increased the likelihood of gaining 

a credential. Sixty-seven percent of the program group members earned a credential, while 
22 percent of the controls did, even though the two groups were indistinguishable in terms of 
motivation, ability and other characteristics. Of the 472 credentials held by sample members, 
44 percent were industry-recognized credentials. The likelihood of earning an industry-rec- 
ognized credential versus a program generated certificate did not differ greatly between the 
program and control groups — 45 percent of the program group's credentials were indus- 
try-recognized, while 39 percent of the controls’ were. 


Table 4 shows the distribution of specific credentials. For both groups, industry-recognized 
credentials are heavily weighted toward A+ certifications as program-generated credentials 
are primarily issued by JVS. It is important to remember that SEIS data is not a nationally 
representative sample that included individuals with all types of credentials but only includes 
credentials obtained by individuals who applied to three specific programs. 


TABLE 3 
CREDENTIALS BY PROGRAM AND CONTROL GROUP 


Program Group 


Control Group 


Full Sample Members Members 
Number % Number % Number % 
Total Sample 1,014 100% 529 100% 485 100% 
Earn Any 472 47% 363 68% 109 22% 
Credential 
Industry 208 44% 165 45% 43. 39% 
Recognized 
Program 264 56% 198 54% 66 60% 
Generated 
Did Not Earn a 542 53% 166 32% 376 78% 
Credential 
TABLE 4 


CREDENTIALS BY PROGRAM AND CONTROL GROUP 


Program Group 


Control Group 


Full Sample Members Members 

Credential Type Number % Number % Number % 
Industry Recognized 208 100% 165 100% 43 100% 
A+ 127 61% 103 62% 24 56% 

CNA 44 21% 31 19% 13 30% 

CMA 23 11% WZ 10% 6 14% 

Construction - Utilities 17 8% 13 8% 4 9% 
Construction- Asbestos 8 4% 6 A% 2 5% 
CDL 3 1% 1 1% 2 5% 

Program Generated 264 100% 198 100% 66 100% 
Medical Office G2 35% 77 40% 13 20% 
Accounting 45 17% 42 21% 3 5% 
Construction-Other 32 13% Zt 14% 6 9% 
Manufacturing 18 7% 17 9% 1 2% 
Clerical/Secretarial 17 6% 6 3% 11 17% 
Other 120 45% 71 36% 49 74% 


Note: Totals of all credentials types may add up to more than the total in each category (industry-recognized and 
program-generated) because some individuals reporting more than one type of credential. 
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2. The earnings effect of sector-focus programs, at least the SEIS programs, appears to 
work through enabling participants to earn credentials. 


Given the large proportion of individuals in the program group that were able to earn creden- 
tials, perhaps the key to programs’ benefits lies in their abilities to help people who otherwise 
would not be able to earn a credential to do so. And, if the effect on earnings works entirely 
through this signaling mechanism -- i.e., obtaining a labor-market valued credential -- sig- 
nals to a local employer that the individual indeed is well trained in skills they need, then we 
should see that once we account for having a credential, there should be no additional earn- 
ings effect for being a member of the program group. 


TABLE 5 


RELATIONSHIP BETWEEN CREDENTIAL RECEIPT AND EARNINGS 
AND HOURS WORKED 24 MONTHS SINCE BASELINE 


Variable Earnings p-value 
Program Status $1,192 0.523 
Credential Status $8,159  ** 0.000 
Sample size 1002 


Note: These estimates are derived from an OLS regression which con- 
trols for an individual's education, gender, race, being U.S.-born, prior 
work history, criminal background, and all the other variables in Table 1. 


The regression results shown in Table 5 confirm this hypothesis. The effect of being a program 
group member is insignificant once we control for having a credential. 


3. The certificates earned by the program group members who were induced by the pro- 
gram to obtain a certificate, on average, increased their earnings by $10,808 over the first 
two years after enrollment. 


If one ignores the experimental nature of this data and just compare the outcomes of peo- 
ple (in both groups) who earned a credential with those that did not, we find that those with 
credentials worked 15.8 months and made $42,902 over the 24 months since random assign- 
ment (i.e., when they applied to one of the programs). Those who did not earn a credential 
worked 11.6 months and earned just $27,530. While this approximate $15,000 difference 
suggests that obtaining a credential leads to better outcomes in terms of earnings and work 
hours, simply comparing outcomes across these two groups is misleading because it does 
not account for the possibility that those who earn credentials could differ systematically from 
those that do not in many ways, such as education, work histories and demographic char- 
acteristics. The results shown in Table 5 shows that almost half of the raw $15,000 earning 
difference is due to the differences in education, gender, race, foreign-born status, prior work 
history, and criminal background (i.e., all the other variables in Table 1) between the creden- 
tial holders and the other individuals. Compared to individuals with similar socioeconomic 
characteristics who did not earn a credential, credential holders earned $8,159 more over the 
two-year period. 


However, this estimate of approximately $8,000 may still be a biased estimate of the impact 
of earning a credential. Simple regression analysis can only control for characteristics that are 
measured. Yet, the two groups of individuals may still be different on unmeasured character- 
istics such as persistence, motivation or aptitude. If motivated and persistent individuals are 
more likely to complete a credential, and these individuals, with or without credentials, would 
have earned more money than less motivated and persistent sample members, then some or 
even all of the $8,000 is not due to the effect of the credential but simply due to the fact that 
they are more persistent and/or able. 


In order to correct for this potential bias, as mentioned above, we use an instrumental vari- 
ables (IV) method to adjust the credential impact estimate for the effects of these unmea- 
sured motivational/ability biases. In particular, we will use the treatment status variable as the 
“instrument” for having a credential at follow-up. How does it work? 


In a random assignment experiment, we know the two groups are, on average, the same 

in all characteristics, including those characteristics that make an individual more likely to 
earn a credential. Thus, the proportion of individuals in the two groups that earn credentials 
“naturally” — i.e., because they are more motivated or persistent — will be the same. Had the 
program group members not attended a sector program, about 22 percent of both groups 
(from Table 3) would have completed a credential and the average follow-up earnings of the 
program group and the control group would have been the same. But as we saw above, 69 
percent of the program group members completed credentials by the follow-up. This higher 
rate of credential completion was not due to the group's higher level of motivation or per- 
sistence (these are the same for the two groups by construction) but to the support the sector 
programs provided. Thus, if the program group on average earned more and the program's 
effect on earnings works entirely through helping participants to earn credentials that are 
valued in the labor market, then we can attribute the difference in earnings to the difference 
in the proportion of the two groups that earned credentials. In particular, the difference in 
earnings would reflect how much more the individuals who would not have “naturally” earned 
a credential benefit from completing a credential. Figure 1 shows how without the program 
the earnings of the two groups would have been the same, while after the program they dif- 
fer to the extent that some of the program groups are induced to complete credentials. 


Figure 1 
Hypothetical Earnings of Sample Members 
Under Different Conditions 
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This IV estimate can be interpreted as the average program effect for individuals who were 
induced to get a credential by participating in the training if three conditions hold: 


1. Attending sector-focused training programs increases every program group member's 
likelihood of earning a credential. This seems quite plausible. 


2. An individual's likelihood of earning a credential did not affect their likelihood of being 
a program group member. This assumption is met by construction since being a mem- 
ber of the program group was determined by a lottery. 


3. The impact of the sector-focused training program must work entirely through the cre- 
dential. In other words, the other aspects of the programs, while supportive of earning a 
credential, do not imbue labor market benefits in and of themselves. Table 5 showed this. 


The first column of Table 6 presents the IV estimate, our best estimate of the impact of en- 
abling an economically disadvantaged individual to earn a credential. For comparison, the 
second column shows the experimental estimate found in Maguire et al. (2010) updated to 
2014 dollars; and the third column reproduces results shown before in Table 5. As the other 
estimates do, the IV estimate controls for observed differences between credential obtainers 
and others, but in addition, to the extent our three assumptions hold, it controls for the unob- 
servable differences between credential holders and non-credential holders (such as motiva- 
tion). We see that among our sample of low-income workers, those that earned credentials 
who would not otherwise have done so without the program earned substantially more than 
they would have without the credential, namely $10,810 over the full two-year period. 


TABLE 6 


THE EFFECTS OF CREDENTIAL RECEIPT ON EARNINGS 
OVER THE 24 MONTHS SINCE BASELINE 


(1) (2) (3) 
IV Experimental OLS 
Variables Estimate Estimate Estimate 
Credential Status $10,810 *** $8,159 *** 
p=.004 p=.000 
Program Status $4,859 *** $1,192 
p=.004 p=.523 
Socio-economic . ‘: . 
controls added 
Sample size 1002 1002 1002 


Note: These estimates derived from an OLS regression in which an individual’s education, 
gender, race, U.S.-born, prior work history, criminal background, and all the other variables 
in Table 1. 


It is not surprising that the IV estimate is larger than the experimental estimate since not all 
participants earned certificates and some of the controls earned credentials too. However, 
somewhat surprisingly, this estimate is also larger than the column 3 estimate where moti- 
vation and other unobservable differences between credential holders and others are not 
controlled ($8,159). How could this be? One potential explanation is that while some of 

the program group members who earned certificates would have done so even without the 
program, these three programs enabled individuals who, on their own without the programs’ 
assistance, would not have had the ability or persistence to complete a credential. These 
individuals receive not only the typical benefit of having a credential but because, without the 
program, they would not have been on this higher trajectory they earn even more than they 
typically would have. 


This means participants of sector-focused training programs who, on their own, would have un- 
derperformed given their age, education, race and other observable characteristics can obtain 
a credential and earn more money than they would have been able to obtain on their own." 


4. Study participants who earned industry-recognized and program-generated credentials 
have similar and significant earnings gains. 


Up to this point, we have treated industry-recognized and program-generated credentials as 
the same, but perhaps their value differs. Thus, next we explore how earnings impacts differ 
between the two types of credentials among study participants. Because we do not have a 
nationally representative sample of low-income individuals, we cannot determine generically 
the value of industry-recognized or program-generated credentials. We have data on only 
individuals who applied to one of the three study programs during the evaluation period. 
Thus, we can estimate the impacts of the industry-recognized credentials and program-gen- 
erated certificates obtained by study participants. 


From Table 4, we know that approximately 60 percent of our designated “industry-recog- 
nized” credentials obtained by our sample members were A+ certifications. However, while 
we know if study participants earned one of six industry-recognized credentials, we cannot 
be sure that the other credentials were only non-industry-recognized. The “program-gener- 
ated” designation is more precisely “program-generated or industry recognition unknown.” 
Among the 264 program-generated certificates, 118 (45 percent) were given by JVS, a 
well-respected training organization in the Boston labor market, for program completion — 
i.e. program-generated. We know little about the other program-generated credentials. 


In our next analysis, we compare the earnings of those with one of the six industry-recognized 
credentials with those of individuals who earned credentials from the other grouping, some 
of whose industry recognition is unconfirmed or are program-generated. This analysis tests 
the hypothesis that industry-recognized credentials are more valuable than non-industry-rec- 
ognized credentials. Given that there is little research on how credentials affect the incomes 
of low-wage workers, this is a valuable step forward. 


"Within the program group, there are individuals who would “naturally” earn a certificate. The earning of those indi- 
viduals — those for whom the program has no effect on their likelihood of earning a certificate — are ostensibly ignored 
in lV analysis. The IV estimate is the effect of enabling someone who in the control group would not have earned a 
credential, to earn one with the program. 
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Table 7 presents the results of the instrumental variable analysis.'2 This analysis controls both 
for the observable and unobservable differences between industry-recognized certificate 
holders, program-generated certificate holders and non-certificate holders. 


When we distinguish between the two types of certificate holders, we find that both types 
of credentials have similar and significant earnings gains over 24 months: $10,802 for the 
industry recognized credential holders and $11,237 for the program-generated group. We 
cannot statistically reject the hypothesis that the two types of credentials have the same 
impact on earnings in our sample. One possible explanation for these findings is that JVS 
completion certificates may function in much the same way as an industry-recognized cre- 
dential does, at least in Boston. However, this finding is likely to be true only in the Boston 
area, since the organization works so closely with Boston's health care employers. 


TABLE 7 


THE INSTRUMENTAL VARIABLES ESTIMATES OF THE IMPACT OF 
OBTAINING AN INDUSTRY-RECOGNIZED CREDENTIAL ORA 
PROGRAM-GENERATED CERTIFICATE ON EARNING OVER THE 
FIRST 24 MONTHS SINCE BASELINE 


Obtaining Obtaining 
an Industry- a Program- 
Obtain Any Recognized Generated 
Outcome Certificate Certificate Certificate 
Earnings Months 
1-24 $10,810: #7" $10,802 ** $11,237 * 
p=.004 p=.049 p=.034 
Socio-economic P, . . 
controls added 
Sample size 1002 1002 1002 


Note: These estimates derived from two stage least square IV regressions in which the three 
program group#site status variables were used as the instrument for the two endogenous vari- 
ables -- obtaining an industry-recognized or a program-generated certificate. In addition the 
regression include an individual's education, gender, race, being U.S.-born, prior work history, 
criminal background and all the other variables in Table 1. 


'2To employ an IV strategy, one needs at least one variable, or instrument, that is correlated with the variable of 
interest — here earning a certificate — but is not correlated with the outcome — earnings or months employed. In 
our case, when we examine the effects of the two types of credentials, we needed at least two variables that were 
correlated with these credential variables (industry-recognized and program-generated) but were uncorrelated 
with unobserved factors that affect our outcomes (employment and earnings). Because the goals of the three 
programs were quite different, there was a variation in the percentage of participants that received each type of 
certificate across the three programs. Thus, we can use the three site-level program-status dummy variables as 
instruments for our two credential variables. 


Is this telling us that all program-generated certificates are equally valuable to all industry rec- 
ognized ones? No, these findings are not saying that all non-industry-recognized credentials 
have impacts of this magnitude. But it does indicate that for the individuals in our sample 
whose credential earning behavior was changed through participation in industry-recognized 
training programs, those that obtained one of the six industry-recognized credentials that we 
asked about earned no more than individuals who received a JVS completion certificate or 
some other credential obtained by sample members. 


CONCLUSIONS AND CAVEATS 


This study is the first to estimate the causal impact on earnings from obtaining a credential 
for low-income job seekers. The results of this analysis indicate that obtaining the types of 
credentials available to our sample members, whether a certificate of completion from a 
well-regarded training organization such as JVS or an industry-recognized credential, boosts 
the earnings of low-income job seekers by about $11,000 in 2014 dollars over two years. This 
is an important finding because low-income job seekers and workers — including those with 
barriers to employment such as welfare receipt, criminal backgrounds and refugee status -- 
are a particularly policy-relevant group of workers. 


It is important to note that three quarters (77 percent) of credential holders in this study were 
program group members who participated in sectoral training programs offered by JVS, WRTP 
or Per Scholas — all well-respected training organizations that had strong, mature relationships 
with local employers. The $11,000 earnings impact estimate is valid as long as the impact of 
these training organizations works entirely through their ability to enable its participants to 
earn credentials. Our test of this assumption (Table 5) confirmed this hypothesis. When the ef- 
fect on earnings was compared between program participants and controls who did not gain 
a credential, on average, program participants without a credential earned about $1,000 more 
over the two years than similar controls who also did not have a credential. But this figure was 
very imprecisely estimated since its likely range was minus $13,000 to plus $15,000.'4 Giv- 

en this range, we cannot reject that being in the program but not getting a credential has no 
effect. However, if this extra non-credential-related advantage of the program were real, then 
perhaps the impact of earning a well-respected credential is somewhat smaller than $11,000 
but still likely substantial. 


The results of our analyses do not support the conclusion that al! industry recognized creden- 
tials or that a certificate of completion issued by any workforce development program will 
significantly impact participants’ employment and earnings. There are three reasons that our 
findings of significant earnings impacts are not generalizable to all credentials: 


1. JVS, WRTP and Per Scholas all collaborated closely with local employers in developing 
and implementing their training. The findings of this study may not characterize the im- 
pact of industry-recognized credentials or certificates of completion provided by less well 
respected organizations on the employment and earnings of low-income job seekers. 


2. The number of different types of non-industry-recognized and industry-recognized cre- 
dentials represented in the SEIS dataset is limited. Indeed, over 27 percent of the ob- 
served credentials were the A+ certifications (and 61 percent of all industry-recognized 
credentials) while 49 percent of the non-industry-recognized in this data set were from 
JVS. This limitation is particularly important given the proliferation and variety of creden- 
tials that industries and training programs are generating, for instance, for retail store sales 
associates or child care providers. 
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The SEIS data set did not include credentials awarded from educational institutions, such as 
community colleges or private colleges. Instead, this study points to the significant role that 
nonprofit organizations can play in working with employers and offering industry-relevant cre- 
dentials. As such, we cannot speak to the influence that the type of institution (educational or 
nonprofit) that awards the certificate has on the size of the impact. 


In addition to examining the earnings impacts of credentials in our sample as a whole, we 
also assessed whether different types of credentials have varied benefits. We found that the 
earnings over time of sample members with industry-recognized credentials were statisti- 
cally similar from those of job seekers with non-industry-recognized credentials, including 
program-generated certificates of training completion. We hypothesize that, at least in the 
three study labor markets, credentials from organizations with strong reputations among local 
businesses may play the same informational role in the local labor market that industry-recog- 
nized credentials (such as A+ or asbestos removal) plays at a national level. But this hypothe- 
sis needs testing. 


FURTHER RESEARCH 


As all good research, these findings beget more questions. What attributes are being sig- 
naled to an employer by the credential? A level of skill? Persistence? Motivation? How do 
those signals influence the employment process? What is the mechanism through which a 
credential overcomes the common employment and earning barriers experienced by wom- 
en, minorities and the formerly incarcerated? While we do not have answers to these ques- 
tions, this research shows that earning a credential provides low-income job seekers with 
some combination of skills and/or endorsement that is valued by employers. And that this 
benefit can be achieved either through a well-regarded industry-recognized credential or a 
certificate provided by a training provider with a strong reputation. The following questions 
are of particular interest: 


e Do trusted training organizations in other local labor markets provide credentials that 
have similar value to employers as the JVS credential? Assuming the presence of similar 
organizations, what leads to their credibility with local businesses? 


e Credentials attained by sample group members were concentrated in IT, construction, 
manufacturing and health care. How does the value of industry-recognized credentials 
vary across occupations and industries and local labor markets? What is the variation be- 
tween licensed and unlicensed occupations/fields? 


e Does receipt of an industry-recognized credential improve employment outcomes re- 
gardless of a candidate’s gender, race/ethnicity or criminal background? For instance, 
does professional certification of a female job candidate in certain industries show similar 
job prospects for a male candidate without certification? What role can credentials play in 
compensating for deficits in education and work experience? 


Answers are needed to these questions for the workforce development field to most effec- 
tively improve job prospects for low-income job seekers. In addition, answers to these ques- 
tions would shed light on employers’ hiring decisions and thus, would help more workforce 
development efforts better prepare their participants for employment. Perhaps most notably, 
these answers would benefit the economic opportunities of low-income job seekers and make 
certain that their investments in postsecondary credentials are valuable in the labor market. 
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